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IMAGE RESOLUTION IMPROVEMENT USING A COLOR MOSAIC SENSOR 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of U.S. 
Provisional Patent Application No. 60/179,954 filed 
February 3, 2000, which is incorporated herein by 
reference . 

FIELD OF THE INVENTION 

The present invention relates generally to imaging 
systems, and particularly to resolution improvement of an 
imaging system comprising a color mosaic sensor. 

BACKGROUND OF THE INVENTION 

Color image sensors comprise a mosaic of individual 
filters covering respective sensor pixels. The filters 
may be red, green and blue (RGB) , or alternatively cyan, 
magenta, yellow and green (CMYG) . After an image has been 
formed on the sensor, signals from adjacent pixels are 
combined so as to produce local color and intensity 
values for the image. Although high-quality video 

cameras use three sensors - one for each primary color - 
mosaic sensors are used in almost all mass-market video 
devices because of their low cost. 

Fig. 1 is a schematic diagram of a "Bayer-type" 
mosaic color imaging sensor, as is known in the art. A 
sensor of this type is the W6500, produced by 
STMicroelectronics of Carrollton, Dallas, Texas. Color 
filters in the sensor are positioned on a rectangular 
grid, there being twice as many G filters as R and B 
filters. To generate an image of a region 12, signals 
from four pixels 14 comprising the region are combined to 
form a color signal having a luminance Y and a 
chrominance C of the region. Values of Y and C are 
calculated as functions, typically linear functions, of 
the signals from the individual pixels 14, i.e., 
1 
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Y = Fl (R, B, Gl, G2) , and 

C = F2 (R, B, Gl, G2) , (1) 
wherein R, B, Gl, and G2 correspond to signals from their 
respective pixels, and Fl and F2 are functions. 

It will be appreciated that the resolution of a 
color imaging sensor is less than that of a black and 
white imaging sensor with the same pixel pitch, since the 
color sensor averages adjacent pixels. 

Figs. 2 and 3 are schematic diagrams showing passage 
of light rays through a lens, as is known in the art. 
Fig. 2 shows white light rays 16, parallel to an axis 20 
of a lens 18, incident on the lens. Because of dispersion 
by the lens, which dispersion is an inherent 
characteristic of all practical lens systems, the 
parallel rays are refracted to different foci on axis 20, 
according to the wavelength, i.e., the color, of the 
dispersed light. Thus, a blue focus 22 is closer to lens 
18 than a red focus 24, and a green focus 26 is 
intermediate between the red and blue foci. Chromatic 
distortions caused because the red, blue, and green foci 
do not coincide on the lens axis are termed axial color 
aberrations . 

Fig. 3 shows a white ray 28, i.e., a ray that exits 
lens 14 (in the absence of aberrations) substantially 
parallel to axis 20, and which thus defines a height of 
an image produced at an image plane 30. As for parallel 
rays 16, ray 28 is dispersed into its constituent colors, 
so causing a chromatic distortion termed lateral color 
aberration at the image plane. 

Other distortions when a lens forms an image are 
also known in the art. For example, a sguare object may 
be imaged with "barrel" or "pincushion" distortion. Also, 
each point on the object will typically be imaged, 
according to a point spread function depending on the 
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lens, to a more or less blurred region of the image 
having an area larger than zero. Methods for correcting 
distortions of the types described above, which are 
typically not functions of the wavelength of the imaging 
light, are known in the art. Examples of methods for 
deblurring images are described in an article titled 
"Review of image-blur models in a photographic system 
using the principles of optics" by Hsien-Che Lee, in the 
May, 1990 issue of Optical Engineering, which is 
incorporated herein by reference. Examples of distortion 
correction are described in an article titled "Digital 
Image Restoration" by Banham et al . , in the March 1997 
issue of IEEE Signal Processing Magazine, which is 
incorporated herein by reference. The book titled 
"Digital Image Restoration" by H.C. Andrews and B.R. 
Hunt, published by Prentice-Hall of Englewood Cliffs, NJ 
in 1977, describes general methods for restoration of 
distorted images . 
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SUMMARY OF THE INVENTION 

It is an object of some aspects of the present 
invention to provide a system and apparatus for improving 
the resolution of images formed by a color mosaic sensor, 
particularly images of documents. 

It is a further object of some aspects of the 
present invention to provide a system and apparatus for 
correction of aberrations generated in an imaging system. 

It is a yet further object of some aspects of the 
present invention to provide a system and apparatus for 
increasing resolution of an imaging system. 

In preferred embodiments of the present invention, 
an imaging system comprises a color mosaic sensor. An 
image is formed on the sensor, which comprises a 
plurality of pixels for sensing different colors, 
preferably red (R) , green (G) , and blue (B) . In at least 
some regions of the sensor, the image formed is 
classifiable into substantially two colors, herein termed 
a background color and a non-background color. This 
binary color characteristic is typical particularly of 
documents. The image is analyzed by a central processing 
unit (CPU) coupled to the sensor in order to determine 
the background and non-background colors. Signals from 
each pixel of the sensor (R, G, and B) are then analyzed, 
and each pixel is re-assigned a color as a function of 
the background and non-background colors. The known 
color in each region of the image makes it possible to 
determine the luminance (Y) levels at each individual 
pixel, and to compare the luminance levels of adjoining 
pixels of different colors. Thus, the resolution of the 
image is enhanced significantly, compared to conventional 
mosaic sensor systems, in which luminance is determined 
only with respect to a group of pixels taken together. 
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In some preferred embodiments of the present 
invention, distortions generated within the imaging 
system are corrected by analyzing signals from each of 
the pixels of the sensor. The system is first calibrated 
with a known object. In operational use a corrected 
signal for each pixel of the sensor is generated by the 
CPU in terms of the calibration. 

In some preferred embodiments of the present 
invention, the resolution of a region of the image is 
improved by analyzing signals from adjacent pixels in the 
region. The analysis is performed on the pixels after 
their colors have been re-assigned as described 
hereinabove, so as to generate an image in the region 
having sub-pixel resolution. 

In some preferred embodiments of the present 
invention, optical character recognition (OCR) is applied 
to images comprising text. The OCR is applied after pixel 
colors have been reassigned and/or after sub-pixel 
resolution has been implemented. 

There is therefore provided, according to a 
preferred embodiment of the present invention, a method 
for electronic imaging, including: 

forming an image including a background color and a 
non-background color on a plurality of pixels in a color 
mosaic image sensor; 

receiving from each of the plurality of pixels a 
respective initial signal responsive to the image; 

determining the background color of the image 
responsive to the initial signals; 

determining the non-background color of the image 
responsive to the initial signals; and 

calculating an adjusted signal for each pixel of the 
plurality of pixels responsive to the initial signal of 
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the pixel and to at least one of the background color and 
the non-background color. 

Preferably, the color mosaic sensor includes pixels 
of at least two specific colors, and determining the 
background color includes locating a background region of 
the image responsive to the initial signals of the pixels 
of at least one of the specific colors, and determining 
the non-background color includes locating a non- 
background region of the image responsive to the initial 
signals of the pixels of the at least one of the specific 
colors . 

Preferably, determining the background color 
includes determining one or more background values 
responsive to the initial signals of the pixels of the at 
least two specific colors in the background region, and 
determining the non-background color includes determining 
one or more non-background values responsive to the 
initial signals of the pixels of the at least two 
specific colors in the non-background region. 

Preferably, calculating the adjusted signal for each 
pixel includes determining the adjusted signal responsive 
to the one or more background values and the one or more 
non-background values . 

Preferably, forming the image includes forming a 
calibration image on the color mosaic image sensor, and 
calculating the adjusted signal for each pixel includes 
determining one or more correction factors for the sensor 
responsive to the calibration image and calculating a 
corrected value for each pixel responsive to the one or 
more correction factors. 

Preferably, calculating the adjusted signal for each 
pixel includes calculating a plurality of sub-pixel 
resolution signals for each pixel responsive to a level 
of the initial signal of the pixel. 

6 
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Further preferably, calculating the plurality of 
sub-pixel resolution signals includes identifying one or 
more straight line segments within the image. 

Preferably, calculating the adjusted signal for each 
pixel includes implementing a process of binarization of 
the image and utilizing the binarization to perform 
optical character recognition (OCR) on at least a portion 
of the image. 

There is further provided, according to a preferred 
embodiment of the present invention, a method for 
electronic imaging, including: 

forming an image including a first plurality of 
areas, each area including a respective background color 
and a respective non-background color, on a second 
plurality of pixels in a color mosaic image sensor; 

receiving from each of the second plurality of 
pixels a respective initial signal responsive to the 
image; 

determining which of the second plurality of pixels 
correspond to each area responsive to the background 
color and non-background color of each area; 

determining for each area the respective background 
color of the image responsive to the initial signals; 

determining for each area the respective non- 
background color of the image responsive to the initial 
signals; and 

calculating an adjusted signal for each pixel of the 
second plurality of pixels responsive to the initial 
signal of the pixel and to at least one of the first 
plurality of background colors and the first plurality of 
non-background colors. 

There is further provided, according to a preferred 
embodiment of the present invention, apparatus for 
electronic imaging, including: 
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a color mosaic image sensor including a plurality of 
pixels, which are adapted to generate respective initial 
signals responsive to an image formed thereon; and 

a central processing unit (CPU) , coupled to receive 
the respective initial signals from the plurality of 
pixels and, responsive to the initial signals, to 
determine a background color and a non-background color 
of the image and to calculate, for each of the plurality 
of pixels, an adjusted signal responsive to the initial 
signal and to at least one of the background color and 
the non-background color. 

Preferably, the plurality of pixels include pixels 
of at least two specific colors, wherein the CPU locates 
a background region and a non-background region of the 
image responsive to the initial signals of the pixels of 
at least one of the specific colors. 

Preferably, the CPU determines one or more 
background values responsive to the initial signals of 
the pixels of the at least two specific colors in the 
background region, and determines one or more non- 
background values responsive to the initial signals of 
the pixels of the at least two specific colors in the 
non-background region. 

Preferably, the CPU determines the adjusted signal 
.responsive to the one or more background values and the 
one or more non-background values. 

Preferably, the apparatus includes a calibration 
grid which forms a calibration image on the color mosaic 
image sensor, wherein the CPU determines one or more 
correction factors for the sensor responsive to the 
calibration image and calculates a corrected value for 
each pixel responsive to the one or more correction 
factors . 
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Preferably, the CPU calculates a plurality of sub- 
pixel resolution signals for each pixel responsive to a 
level of the initial signal of the pixel. 

Further preferably, the CPU determines one or more 
straight line segments within the image. 

Preferably, the CPU implements a process of 
binarization of the image and utilizes the binarization 
to perform optical character recognition (OCR) on at 
least a portion of the image. 

There is further provided, according to a preferred 
embodiment of the present invention, apparatus for 
electronic imaging, including: 

a color mosaic image sensor including a first 
plurality of pixels, which are adapted to generate 
respective initial signals responsive to an image, 
including a second plurality of areas, each area 
including a respective background color and a respective 

non-background color, formed thereon; and 

a central processing unit (CPU) , coupled to receive 
the respective initial signals from the first plurality 
of pixels and which is adapted, responsive to the initial 
signals, to determine which of the pixels correspond to 
each area responsive to the background color and non- 
background color of each area, to determine for each area 
a background color and a non-background color of the 
image, and to calculate, for each of the first plurality 
of pixels, an adjusted signal responsive to the initial 
signal and to at least one of the second plurality of 
background colors and the second plurality of non- 
background colors. 

The present invention will be more fully understood 
from the following detailed description of the preferred 
embodiments thereof, taken together with the drawings, in 
which : 

9 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram of a color imaging 
sensor, as is known in the art; 

Fig. 2 is a first schematic diagram showing passage 
of light rays through a lens, as is known in the art; 

Fig. 3 is a second schematic diagram showing passage 
of light rays through a lens, as is known in the art; 

Fig. 4 is a schematic diagram of an imaging system, 
according to a preferred embodiment of the present 
invention; 

Fig. 5 is a flowchart illustrating a process for 
improving the resolution of an image captured in the 
imaging system of Fig. 4, according to a preferred 
embodiment of the present invention; 

Fig. 6 is a schematic diagram of a calibration grid 
used in a process for reducing effects of lens 
aberrations, according to a preferred embodiment of the 
present invention; 

Fig. 7 is a schematic diagram showing an offset 
image, according to a preferred embodiment of the present 
invention; 

Fig. 8 is a schematic diagram illustrating 
generation of an image at sub-pixel resolution, according 
to a preferred embodiment of the present invention; 

Fig. 9 is a flowchart showing a method for 
generating sub-pixel resolution, according to a preferred 
embodiment of the present invention; and 

Fig. 10 is a flowchart showing a method for optical 
character recognition (OCR) , according to a preferred 
embodiment of the present invention; 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Reference is now made to Fig. 4, which is a 
schematic diagram of an imaging system 40, according to a 
preferred embodiment of the present invention. Imaging 
system 40 comprises an imaging device 42 which forms an 
image of a document 46 on a color imaging sensor 44. 
System 40 also comprises a central processing unit (CPU) 
54, most preferably any industry-standard CPU, which 
receives signals from sensor 44 and analyzes the signals, 
as described in detail hereinbelow. Sensor 44 comprises 
any industry-standard color mosaic sensor, such as the 
Bayer-type sensor described in the Background of the 
Invention. Device 42 comprises one or more optical 
elements known in the art, which are able to form 
separately or in combination an image of an object on 
sensor 44. Typically, device 42 comprises one or more 
simple or compound lenses, and/or one or more mirrors. 
Device 42 may be implemented from industry-standard 
elements, or custom-built elements, or a combination of 
industry-standard and custom-built elements. 

As described in the Background of the Invention, a 
luminance Y and a chrominance C of a region 48 of 
document 4 6 are given by the equations: 
Y = Fl (R, B, Gl, G2) , and 

C = F2 (R, B, Gl, G2) , (D 
wherein R, B, Gl, and G2 correspond to signals from 
pixels of sensor 44, generated by the image of region 48 
on an area 50 of the sensor. If region 48 is assumed to 
be substantially of one known color, then substantially 
fixed ratios 

aB= t' aR= l (2) 

will prevail in the region. Knowledge of values of oc B and 
a R can be used to calculate a value of Y and C on a 
11 
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pixel-by-pixel basis. For example, if a specific red 
pixel 52 gives a signal R s , then calculated values of a 
green signal G s and a blue signal B s at pixel 52 are 
given by, 

G s =*i-, Bs =^Rs (3) 

Substituting (R s , B s , G s ) into equation (1) 
(assuming Gl = G2 = G s ) gives a value for Y and C at 
pixel 52. Similarly, using values of a B , a R enables 
values of Y and C to be made for other pixels in region 
48. 

In document imaging, it is typical that 
substantially all regions in the document are 
classifiable as having one of two colors predominating in 
the document, termed herein a background color and a non- 
background color. For example, document 4 6 may comprise 
black text on a white background, or a red line drawing 
on a pale yellow background. Preferred embodiments of the 
present invention are able to function with an image 
comprised of substantially any pair of colors. 
Hereinbelow, signals from the background color are 
identified with a subscript "b," and signals from the 
non-background color are identified with a subscript "n." 

In a region which is known to be predominantly 
background, ratios comprising red signals (R b ) , green 
signals (G b ) , and blue signals (B b ) from their respective 
pixels will be substantially constant over the region. 

Thus a ratio — has a substantially constant value, 
G b 

herein termed oc Bb , and a ratio — — has a substantially 

constant value, herein termed a Rb , for signals from the 
background region. Similarly, in a region which is known 
to be predominantly non-background, ratios for red 
12 
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signals (R n ) , green signals (G n ) , and blue signals (B n ) 
will also be substantially constant. Thus there is a 

substantially constant value ag n , equal to a ratio , 

G n 

and a substantially constant value a.R n/ equal to a ratio 
R n 

— — for signals from the non-background region. 
G n 

Fig. 5 is a flowchart illustrating a process for 
improving the resolution of an image captured in system 
40, according to a preferred embodiment of the present 
invention. In an initial step, an image of document 46 is 
formed on sensor 44 of imaging system 40 (Fig. 4) . 

In a background analysis step, signals from pixels 
in sensor 44 are analyzed to determine one or more 
regions where the image comprises mainly background 
color. The analysis is most preferably implemented by 
finding regions where the change in signal level from a 
specific color (R, G, or B) pixel to a closest-neighbor 
same-color pixel is relatively low, indicating that the 
region being imaged comprises substantially one color. 
Averages, herein termed Rb , Gb , and Bb , of all 
respective Rb , Gb , and Bb signals within such a region 
are calculated. The averages are used to calculate 
background values of and a^: 

R b B b 

a Rb = =, «Bb== ( 4 ) 

G b G b 

In a non-background analysis step, signals from 
sensor 44 are analyzed in generally the same manner as 
described for the background analysis step. The non- 
background regions are chosen from substantially single- 
color regions which have significantly different signal 
values from those for pixels in the background regions. 
Averages, herein termed R n , G n , and B n , of all 
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respective R n/ G n , and B n , signals within such a region 
are calculated. The averages are used to calculate non- 
background values of a^h and ag n : 

aRn==, *Bn== (5) 
G n G n 

In cases where non-background regions are not 
sufficiently large or well-defined, for example in the 
case when the non-background regions are text, most 
preferably vertical and horizontal parts of specific 
letters are used to calculate a^ n and ag n . Methods for 
determining such parts, for example by comparing 
orthogonal signal gradients, are well known in the art. 

Each pixel of sensor 44 images a color which is 
substantially background, non-background, or a 
combination of background and non-background, and 
generates a signal "x." In a final step, signal x and 
values of a^, a^, ap Lri/ and oc Bn are used to generate an 
intensity and a color for each pixel. Most preferably, a B 
and ccr for each pixel are calculated using a linear 

combination of ^^k. and the linear combination 

a Rb a Rn 

being a function of how x compares to averaged background 
and non-background values for the specific pixel. These 
values are then used in equations (3) and (2), as 
necessary, in order to find values of R, G, and B to 
substitute into equation (1), and so find a value of Y 
and C for each pixel. 

Fig. 6 is a schematic diagram of a calibration grid 
60 used in a process for reducing effects of lens 
aberrations, according to a preferred embodiment of the 
present invention. Grid 60 is most preferably implemented 
as a set of transparent horizontal and vertical lines 62 
on a substantially opaque background 6,4, the lines 
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forming a substantially square array. Lines 62 are most 
preferably blurred at their edges, so as to prevent, as 
explained in more detail below, aliasing of an image 
produced by grid 60. Grid 60 is dimensioned so that each 
of lines 62 forms an image more than two pixels wide on 
sensor 44, when the grid is positioned as object 46 in 
system 40 (Fig. 4) . To image grid 60 on the sensor, the 
grid is most preferably illuminated by substantially 
white light. 

Each color plane of sensor 44 is analyzed 
separately, to locate vertices in the image of the grid. 
The vertices are located by detecting vertical and 
horizontal line crossings. Thus, considering the blue 
plane, all blue pixels generating a signal above a 
predetermined level are analyzed. The pixels are 
allocated to a specific vertical and/or a specific 
horizontal line structure, by determining relationships 
between adjacent blue pixels, or by other means known in 
the art. A first order determination is made of a vertex 
as one or more adjacent blue pixels which have been 
allocated to both a vertical and a horizontal line 
structure. A more accurate determination of the position 
of the particular vertex is made by finding the point of 
intersection of the vertical and horizontal lines in the 
region of the first order vertex. Preferably, the 
coordinates of each vertex (x n , y n ) are determined by 
finding a weighted mean of signals B in the region of the 
vertex, according to the following equations: 

Aliasing in summations in equations (6) is 
substantially eliminated by the blurring of lines 62. 
Similar analyses are implemented for the red and green 
planes . 
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As explained in the Background of the Invention with 
reference to Figs. 2 and 3, images formed by device 42, 
particularly when the device is a simple lens, will have 
chromatic aberrations. Thus, the location of any specific 
vertex, as determined by equations (6) for the three 
color planes, will generally be displaced one from the 
other. Typically the blue vertices will be displaced 
towards the center of the image, and the red vertices 
will be displaced away from the center of the image, 
relative to the green vertices. To compensate for the 
offset, correction factors are calculated for each red 
vertex and each blue vertex, as described herein. 

The horizontal offset for each red vertex is assumed 
to be expressed as a two-dimensional second order 
polynomial, i.e., 

Ax R n = a 0 XR n 2 + a iyRn 2 + a 2 x Rn • y Rn + a 3 x Rn + a 4 y Rn + a 5 ( 7 ) 
wherein Ax Rfl is the offset, and (x Rn , y Rn ) are the 
coordinates, of the nth red vertex, and a 0 , a lf a 2 , a 3 , 
a^, a5, are correction coefficients for sensor 44. 

If grid 60 is assumed to comprise L vertices, there 
will thus be L simultaneous equations (2) with six 
unknowns. The L equations may be written in matrix 
format : 

AXRn = x R n a (8) 

The equations are used to evaluate values for ag, a]_, a 2 , 
a3, a4, and a$, preferably by a method of least squares 
fitting, or alternatively by some other standard method 
known in the art for solving simultaneous equations. The 
values of ag, a]_, a 2 , a3, aq, and a5 are then stored in 
CPU 54. A similar process to that described above is 
implemented in order to find corresponding correction 
coefficients for second-order polynomials for the 

vertical offset of each red vertex. The process is also 
16 
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implemented for the horizontal and vertical offsets of 
each blue vertex, so that a total of 24 correction 
coefficients are generated and stored in CPU 54 . These 
coefficients are used, as described hereinbelow, to 
improve the color balance of an image formed on sensor 
44 . 

Fig. 7 is a schematic diagram showing an offset 

image 70, according to a preferred embodiment of the 

present invention. Image 70 on a blue pixel 68 is offset 

from the pixel because of distortions which are 

introduced by imaging system 40. Image 70 is offset 

horizontally by Axg and vertically by Ayg. Thus the 

signal B generated by pixel 68, and the corresponding 

green value G(B) based on B, due to image 70, is reduced. 

A corrected value G(Bcor) of signal G(B) is preferably 

calculated as follows: 

Ay B Gl + Ax B -G2 + G(B) 

^K^cor) 7 7 ( y ) 

Ay B + Ax B +1 

wherein Gl, G2, and G(B) correspond to respective 
signals, generated at pixels 72, 74, and 68, which have 
been corrected according to equations (1), (2), and (3) 
as described above. 

Equation (9) is generated as one possible form of a 
linear interpolation using signal values at pixels 72, 74 
and 68. Equation (9) assumes that only nearest-neighbor 
vertical and horizontal pixels to pixel 68 have a 
substantial effect on signal B. 

Alternatively, a corrected value G(B cor ) is 
calculated as follows : 

= G(B) - Ax B (l - Ay B )G3 - Ay B C - Ax B )G4 - Ax B Ay B G(R4) 
(l-Ax B )(l-Ay B ) 

wherein G3, G4, and G(R4) correspond to respective 
signals, generated at pixels 76, 78, and 80, which have 
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been corrected according to equations (1), (2), and (3) 
as described above. 

Equation (10) is a linear interpolation of signal 
values from all nearest-neighbor pixels which receive 
energy because of the offset of image 70. 

Values of Ax B , Ay B for pixel 70 are calculated from 
the "blue" correction coefficients, determined as 
described above with reference to Fig. 6, stored in CPU 
54. A similar process is applied to generate corrected 
signals for the other blue pixels, and for the red 
pixels, of sensor 44, using the appropriate correction 
coefficients in each case, so as to improve overall image 
color balance produced by the sensor. 

In some preferred embodiments of the present 
invention, the image quality is further improved by 
enhancing edges of the image. Edges are determined by 
considering gradients of specific color plane pixels. For 
example, if a pixel is found to be substantially in 
alignment with an edge, its contribution to equations (9) 
and (10) is enhanced. If the pixel is substantially 
orthogonal to the edge, its contribution to the equations 
is reduced. Methods for determining edges of images are 
well known in the art. 

Fig. 8 is a schematic diagram illustrating an image 
which is generated to sub-pixel resolution, according to 
a preferred embodiment of the present invention. A 
vertical line 90, having a width of 2 pixels and having a 
non-background color, is imaged onto sensor 44. Line 90 
is assumed to be aligned with a vertical axis of the 
sensor. Line 90 is part of an image 91, substantially 
comprised of non-background and background colors. A 
pixel which completely images non-background color is 
assumed to generate a signal equal to 255. A pixel which 
completely images background color is assumed to generate 
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a signal equal to 0 . An ideal output from three pixels 
Al, A2, and A3 is shown in a graph 92, wherein a width of 
pixel Al is W]_, and a width of pixel A3 is W3, where W]_ 
and W3 are fractions of a width of a pixel in sensor 44. 
It is assumed that W]_ > 0.5, and that W 3 < 0.5. An actual 
output from the three pixels Al, A2, and A3 is shown in a 
graph 94. 

A graph 96 shows the effect of doubling the pixel 
resolution to sub-pixels Bl, B2, B6. The signal from 

pixel Al is divided into two, a first signal at sub-pixel 
B2, closest to the known non-background value of A2, 
having a value 255, and a second signal at sub-pixel Bl 
having a value 2* (W]_-0 . 5) *255 . Similarly, the signal from 
pixel A3 is divided into two, a first signal at sub-pixel 
B5, closest to the known non-background value of A2, 
having a value 2*W 3 *255, and a second signal at sub-pixel 
B6 having a value 0. 

A graph 98 shows the effect of further doubling the 
pixel resolution to sub-pixels CI, C2, CIO, and 

allocating the signal of each sub-pixel to be either 255 
or 0, depending on which value is closest after the 
doubling process. Thus, sub-pixel Bl is split into a sub- 
pixel C2 having a signal 255, and a sub-pixel CI having a 

signal 0, assuming that — < Wi < -J™ . Similarly sub-pixel 
255 255 ^ 

B5 is split into a sub-pixel C9 having a signal 255, and 
a sub-pixel C10 having a signal 0, assuming that 

— <w 3 <^. 

255 3 255 

The process illustrated in Fig. 8 is theoretical, 
and has assumed that line 90 is aligned with an axis of 
sensor 44. In practice, a line that is imaged on sensor 
44 will in general not be so aligned. Also in practice, 
imaging system 40 (Fig. 4) spreads the image of each 
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point of object 46, according to a point spread function, 
as is known in the art (the spreading function is 
typically dependent on properties of imaging device 42, 
such as an effective aperture of the device) . 
Furthermore, object 4 6 is typically not only comprised of 
lines such as line 90. The method described hereinbelow, 
with reference to Fig. 9, substantially resolves these 
limitations . 

Fig. 9 is a flowchart showing a method for 
generating sub-pixel resolution, according to a preferred 
embodiment of the present invention. The method shown in 
Fig. 9 implements the process described above with 
reference to Fig. 8. In initial steps, an image is formed 
on sensor 44, and an image resolution improvement 
process, substantially as described above with reference 
to Fig. 5, is performed. The image is assumed to be 
implemented substantially from two colors, and 'elements 
of the image which are to have their resolution increased 
are assumed to comprise substantially linear elements. 

In an analysis step, the image is analyzed for 
single straight line segments formed from pixels having a 
non-background color, by methods known in the art. 
Typically, line segments found will not be aligned with 
an axis of the sensor. In these cases, a linear 
transformation of pixels comprising each segment, and of 
pixels in the region of each respective segment, is 
performed. For each segment, the transformation is 
performed so as to generate a set of eguivalent pixel 
values on a grid having an axis perpendicular to the 
respective line. Such linear transformations are well 
known in the art . 

Regions which have been found to comprise single 
straight line segments are then deblurred, using one of 
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the deblurring algorithms known in the art. The specific 
deblurring algorithm generates deblurred pixels. 

After necessary transformations and deblurring, a 
process of sub-pixel resolution substantially similar to 
the process described above with reference to Fig. 8 is 
performed on each region of deblurred pixels. 

It will be appreciated that other methods for 
generating sub-pixel resolution are known in the art. 
Such methods are most preferably implemented using 
preferred embodiments of the present invention as 
described hereinabove. It will also be appreciated that 
during the correction process described hereinabove with 
reference to Fig. 7, a pixel-sized image is offset and so 
overlaps more than one pixel. These pixels are analyzed 
more than once, according to the colors of adjacent 
pixels. For example, information regarding pixel G3 can 
be found from G3 itself and from the overlap of pixel B 
on G3 . Those skilled in the art will be able to correlate 
these analyses to further increase the resolution of 
sensor 4 4 . 

Fig. 10 is a flowchart showing a method for optical 
character recognition (OCR) , according to a preferred 
embodiment of the present invention. As is known in the 
art, OCR systems use a system of binarization of an image 
before processing the image to derive text. The OCR 
system performance is very sensitive to the binarization 
process used. The flowchart described herein uses gray 
level information and knowledge of adjacent pixel 
interrelationships generated by sensor 44, together with 
a knowledge of the point spread function of imaging 
device 42, to improve OCR. 

In a first step, an image comprising text is formed 
on sensor 44. Most preferably, the image is deblurred 
using one of the deblurring algorithms known in the art. 
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The image is then processed substantially according to 
the method described hereinabove with reference to Fig. 
5, in order to improve the resolution of the image. The 
improved image is then processed substantially according 
to the methods described hereinabove with reference to 
Figs. 7, 8 and 9, to improve image color balance, and to 
resolve the image to sub-pixel resolution. Finally, the 
image is binarized and subjected to an OCR process known 
in the art. 

Referring back to Figs. 4 and 5, in some preferred 
embodiments of the present invention the process of Fig. 
5 is adapted to restore delicate lines in an image. If 
image 50 comprises fine lines having the non-background 
color on the background color, the fine lines may be 
initially undetected because pixels comprising the fine 
lines are assumed to be part of the background. However, 
once the non-background color has been determined, a 
later analysis can restore the fine lines by "forcing" 
the color. For example, assume the non-background color 



has been determined to be cyan, . A green pixel 



which initially does not appear to be cyan, but which has 
adjacent blue pixels which are cyan, may be designated as 
being color cyan. The signal of the green pixel is 
altered accordingly. Forcing a color to a non-background 
pixel (according to its relevant neighbors) helps restore 
the line, and also prevents color artifacts that normally 
characterize images of lines taken by a mosaic sensor. 

In some preferred embodiments of the present 
invention, system 40 is adapted to be used as facsimile 
machine. Most preferably, one or more of the methods 
described hereinabove for improving image quality of 
sensor 44 are implemented so as to improve the quality of 
fax images transmitted by the machine. Similarly, system 
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40 can be adapted as a business card reader, preferably 
comprising OCR, so that the image and/or data related to 
a business card can be read and stored in a database. In 
some preferred embodiments of the present invention, 
system 40, together with OCR, is used as a document or 
phrase or word translator, using automatic or semi- 
automatic translation methods known in the art. 

In some preferred embodiments of the present 
invention, system 40 is adapted for use as a barcode 
reader, most preferably by further implementing the sub- 
pixel resolution improvement described above with 
reference to Fig. 8. 

In some preferred embodiments of the present 
invention, most preferably when the object being imaged 
comprises text, system 40 is used as an image data 
compression device. The object comprising text is first 
imaged, and the image is processed via OCR to generate 
text. The text is transmitted or stored as is, and, if 
appropriate, is further processed via a text compression 
method known in the art. If a grayscale object is found 
in the object, the image is preferably compressed to a 
binary level by one of the processes known in the art, 
such as dithering or half-toning. 

In some preferred embodiments of the present 
invention, system 40 is adapted to be able to operate 
either as a full-color imaging system, i.e., without 
implementing the resolution improvement methods described 
hereinabove, or as one or more of the resolution 
improvement systems described hereinabove. 

It will be appreciated that preferred embodiments of 
the present invention are able to function with an image 
comprised of more than one area, each area being 
comprised of a pair of colors generally different from 
pairs of colors of other areas of the image. For example, 
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a document which is imaged may be comprised of a first 
area having black text on a white background, and a 
second area having a blue line drawing on a yellow 
background. After determining the first and second areas, 
by methods known in the art, each area may be operated on 
in a generally independent manner, as described 
hereinabove . 

It will thus be appreciated that the preferred 
embodiments described above are cited by way of example, 
and that the present invention is not limited to what has 
been particularly shown and described hereinabove. 
Rather, the scope of the present invention includes both 
combinations and subcombinations of the various features 
described hereinabove, as well as variations and 
modifications thereof which would occur to persons 
skilled in the art upon reading the foregoing description 
and which are not disclosed in the prior art. 
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